chore: nits

chore: add mockstore

refactor: testdb

chore: remove into_hashtype dup

refactor(db): `TestDb` constructor (#1351)

I recently learned from @demosdemon that in Rust, the default
constructor of a type is generally called `new()` while additional
constructors are called `with_X()`.

This PR refactors the `TestDb` code to have a `new()` constructor with
this function being called through the database tests instead of
`testdb()`. This helps break up #1346 as we'll just then need to add a
`with_mockstore()` method to `TestDb` to allow for injecting instances
of `MockStore` to `TestDb`.

feat(monitoring): Grafana automatic dashboard and auth provisioning (#1307)

This PR modifies the Grafana installation script to automatically
provision the datasource and a dashboard directory. It also modifies the
dashboard file with a configurable datasource so it automatically picks
up the available prometheus one.

fix: explicitly release advisory lock on drop (#1352)

This change ensures that the advisory lock is explicitly released when
the `FileBacked` struct is dropped, preventing an issue where we would
race on reopening the file and acquiring the lock again.

Stress testing over 20 minutes has confirmed that this resolves the
issue where previous attempts would trigger the bug within 2 minutes.

Fixes: #1250

chore(nodestore): remove empty committed err (#1353)

While factoring out the `NodeStore` changes of `RootStore` into its own
PR, I found that `new_empty_committed()` returns a `Result` type but
always returns `Ok`. This PR simplifies `new_empty_committed()` by
removing the `Result` return type.

chore(nodestore): access persisted nodestores

feat(ffi): implement revision handles and expose to FFI layer (#1326)

This PR introduces revision handles that hold a revision across the FFI
while you use it, so it won’t be removed mid-operation. This should
improve concurrency and avoids errors as mentioned in #1008 . It also
makes iterator's design simpler, we can create them from a held
RevisionHandle with no tricky lifetimes.

This updates the existing Go bindings so `Database.Revision` returns a
Revision with `Get` and `Drop` methods. Necessary changes to tests are
applied.

Closes #1008.

chore: nit

feat(ffi-iterator): Implementation of Iterator in Rust (1/4) (#1255)

This PR is a step towards implementing the FFI iterator mentioned in

This implement FFI functions (`fwd_iter_on_root`,
`fwd_iter_on_proposal`, `fwd_iter_next`) in Rust, and also necessary
changes in `firewood` crate for `MerkleKeyValueStream` to work with
`Arc<NodeStore<..>>` as well.

---------

Co-authored-by: Brandon LeBlanc <[email protected]>

feat(ffi-iterator): Implementation of Iterator in Go (2/4) (#1256)

This implement a simple Iterator interface in Go (`Next`, `Key`,
`Value`) using the introduced FFI functions, without batching.

Depends On: #1255

---------

Co-authored-by: Brandon LeBlanc <[email protected]>

Revert "feat(ffi-iterator): Implementation of Iterator in Go (2/4) (#1256)"

This reverts commit 9dd02f1.

Revert "feat(ffi-iterator): Implementation of Iterator in Rust (1/4) (#1255)"

This reverts commit e44391c.

chore: with_root

chore: with_root

chore: move storage ops into with_root

chore: rename test

chore: debug_assert_eq

Revert "Merge branch 'rodrigo/add-committed-nodestore-constructor' into rodrigo/add-rootstore-to-manager"

This reverts commit 5d4defa, reversing
changes made to 7da0e85.

fix: EINTR during iouring calls (#1354)

golang does not guarantee signals are using SA_RESTART, so any system
call on a "slow device" might cause the EINTR error. We saw this in the
error in issue 1347.

This doesn't resolve that issue completely because IO errors can occur
during these writes. That will be fixed under a separate issue.

My research indicated that re-calling the system call is the right thing
to do here.

feat(1/7): add U4 type to be used as a path component (#1336)

This change introduces a `U4` type that uses an underlying enum with
only 16 states in order to enable niche optimizations. This change does
not yet use the type within code as that will cause a large amount of
changes that will make reviewing this difficult. The focus of this
change is the `U4` type itself and nothing outside of that. This type
also is not a `PathComponent`, which will appear in the next Pull
Request.

The niche optimizations apply to memory layout allowing for `Option<U4>`
to have the same size and alignment of `U4`, meaning there is no
additional storage to keep the Option discriminant.

The niche optimizations also apply to bounds checking when indexing into
a fixed size array, like we do for the `Children<_>` type alias. See
<https://rust.godbolt.org/z/cqdz1n9d5>

This change contributes to the resolution of #1230.

feat(2/7): add newtype for PathComponent (#1337)

The new `PathComponent` type is introduced to encapsulate the possible
values of a single component in the Trie path. If `branch_factor_256` is
not enabled, the default, `PathComponent` wraps `U4`, which was added in
the previous pull request. This type is only value for values 0-15,
inclusive. If `branch_factor_256` is enabled, `PathComponent` wraps a
`u8`, which can represent values 0-255, inclusive.

This change contributes to the resolution of #1230, but does not fully
resolve it until all usages of `Path` are replaced with future changes.

feat(3/7): add TriePath trait and path abstraction (#1338)

The `TriePath` trait provides a unified interface for handling paths in
the trie structure. Somtimes paths are backed by a sequence of bytes,
and sometimes by a sequence of components. For branch_factor_256, they
are the same. This abstraction allows us to work with both without
running into type issues and inconsistencies with the two
representations.

feat(4/7): add SplitPath trait (#1339)

One of the most common operations on paths is finding the longest common
prefix between two paths. This is useful for a variety of reasons, such
as determining if one path is a subpath of another, or finding the
relative path between two paths.

A normal `TriePath` implementation does not provide a way to do this
because that trait should remain `?Sized`. Whereas, splitting the path
requires a sized type in order to produce a new instance of the path
when splitting.

This change adds a `SplitPath` trait that provides the ability to split
a path. Also added is `IntoSplitPath` which is a conversion trait to
yield a splittable path. This also acts as the borrower for owned paths
to yield shared references to the path.

chore: upgrade dependencies (#1360)

The transitive upgrade on `generic-array` caused a large number of
deprecation warnings and is beginning to interfere with CI on pull
requests. To resolve this, I replaced the generic-array with a plain
array in TrieHash and annotated the remaining instances with the lint
expectation.

chore(nodestore): access persisted nodestores (#1355)

This PR breaks up #1346 by factoring out the `NodeStore` changes.

This PR allows for persisted nodestores which are no longer in memory to
be retrieved again. This is useful for `RootStore` as this is what it'll
use to construct prior revisions in the db `view()` method.

feat(ffi-iterator): Implementation of Iterator in Rust (1/4) (#1255)

This PR is a step towards implementing the FFI iterator mentioned in

This implement FFI functions (`fwd_iter_on_root`,
`fwd_iter_on_proposal`, `fwd_iter_next`) in Rust, and also necessary
changes in `firewood` crate for `MerkleKeyValueStream` to work with
`Arc<NodeStore<..>>` as well.

---------

Co-authored-by: Brandon LeBlanc <[email protected]>

feat(ffi-iterator): Implementation of Iterator in Go (2/4) (#1256)

This implement a simple Iterator interface in Go (`Next`, `Key`,
`Value`) using the introduced FFI functions, without batching.

Depends On: #1255

---------

Co-authored-by: Brandon LeBlanc <[email protected]>